<html>
<head><meta charset="utf-8"><title>Running all tests in PR · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html">Running all tests in PR</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="216819552"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216819552" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216819552">(Nov 15 2020 at 22:56)</a>:</h4>
<p>continuing the thread from <a href="https://github.com/rust-lang/rust/pull/78801#discussion_r523728344">https://github.com/rust-lang/rust/pull/78801#discussion_r523728344</a></p>
<p>I'm wondering how does one run all the tests (similar to Pr merge process) when creating a PR?  This is I guess more for future now but curious</p>



<a name="216821533"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821533" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821533">(Nov 15 2020 at 23:54)</a>:</h4>
<p>Are you asking how to replicate the tests locally?</p>



<a name="216821540"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821540" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821540">(Nov 15 2020 at 23:54)</a>:</h4>
<p><code>x.py test --stage 2</code> catches most things</p>



<a name="216821545"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821545" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821545">(Nov 15 2020 at 23:54)</a>:</h4>
<p>That's usually more than you need, though</p>



<a name="216821613"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821613" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821613">(Nov 15 2020 at 23:57)</a>:</h4>
<p><span class="user-mention" data-user-id="281950">@Aman Arora</span>  If you mean running all tests locally, that's not really practical since there are 60 builders which take about 2 hours to run, which would take about 5 days to run serially. <span aria-label="grinning" class="emoji emoji-1f600" role="img" title="grinning">:grinning:</span>  The PR "check" that runs every time you push to a PR runs <code>x.py test</code>.  It only tests the tools if the PR modifies one of the submodules (which now that clippy is no longer a submodule, means it doesn't normally get checked). To test something like clippy, it would be <code>./x.py test src/tools/clippy</code>.  <a href="https://github.com/rust-lang/rust/blob/master/src/ci/docker/host-x86_64/x86_64-gnu-tools/checktools.sh">Here</a> is the list of other tools that normally run tests.</p>
<p>It might be interesting to figure out how often a PR fails just due to clippy.  If it is often enough, it might be worthwhile to add it to the PR check.</p>



<a name="216821677"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821677" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821677">(Nov 15 2020 at 23:59)</a>:</h4>
<p>Locally I run <code>test --stage 1</code>. I was asking on Github actions/CI that is used on rust-lang. Essentially I felt like my PR might break some things and thought a bors try would catch anything that would break during the merge process.</p>



<a name="216821766"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821766" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821766">(Nov 16 2020 at 00:00)</a>:</h4>
<p>But apparently based on the comment on the thread, bors try doesn't run all if any tests.</p>



<a name="216821775"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216821775" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216821775">(Nov 16 2020 at 00:01)</a>:</h4>
<p>So I'm curious if we have a  bors command or how could one run the test suite (before merge) by editing buildbot files (as mentioned in the thread)</p>



<a name="216822046"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216822046" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216822046">(Nov 16 2020 at 00:08)</a>:</h4>
<p>There isn't a command to run more tests.  In rare cases, people may edit the <a href="https://github.com/rust-lang/rust/blob/603ab5bd6e0ffefafa7411cd8bd23a6ca82bcff0/.github/workflows/ci.yml#L40-L52">ci.yml</a> file to add an entry to the PR job to test something specific.  I personally just run tests in docker locally (documented <a href="https://rustc-dev-guide.rust-lang.org/tests/intro.html#testing-with-docker-images">here</a>) if I have a hunch it will break on specific platforms.  Otherwise, if it seems like a risky PR, just mark it <a href="https://forge.rust-lang.org/compiler/reviews.html#rollups">iffy</a>, and let it go through the queue.</p>



<a name="216822845"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216822845" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216822845">(Nov 16 2020 at 00:32)</a>:</h4>
<p>I see, thanks!</p>



<a name="216839769"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216839769" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216839769">(Nov 16 2020 at 07:38)</a>:</h4>
<p>(no need to mark 78801 iffy btw, it’s already rollup=never) and for even subtler/riskier cases, like ones where the current tests are not enough, there’s also crater to try to compile everything on <a href="http://crates.io">crates.io</a> (or run those crates’ tests) with the PR artifacts</p>



<a name="216949414"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Running%20all%20tests%20in%20PR/near/216949414" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aman Arora <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Running.20all.20tests.20in.20PR.html#216949414">(Nov 16 2020 at 23:52)</a>:</h4>
<p>Maybe run crater once we feature is ready <span aria-label="sweat smile" class="emoji emoji-1f605" role="img" title="sweat smile">:sweat_smile:</span>. Also had a feeling clippy might break but wasn't able to run test locally (now have it figured out) and was just relying on bors try to avoid having to figure it out lol</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>